Peripheral Component Interconnect (PCI) Express is a standard for coupling motherboard-mounted peripherals, and as an expansion card interface for add-in boards. Typically, PCI Express (PCIe) peripherals are auto configured via a PCI configuration space. In addition to memory-mapped and I/O port spaces, each device on the PCIe bus has a configuration space.
In a PCI Express system, a root complex device typically couples a processor and memory subsystem to a PCI Express switch fabric comprising one or more switch devices. The PCI Express system may also include endpoints configured to perform and/or request PCI Express transactions. Each endpoint typically comprises one or more functions, and is mapped into the configuration space as a single function in a device that may include either the single function or multiple functions.
PCI Express endpoints and legacy (i.e., PCI) endpoints typically appear within a hierarchical domain originated by the root complex. In other words, the endpoints appear in the configuration space as a tree, with a root port as its head. Additionally, root complex integrated endpoints and root complex event collectors typically do not appear within one of the hierarchical domains originated by the root complex. Instead, the root complex integrated endpoints and root complex event collectors generally appear in the configuration space as peers of the root ports.
Implementing virtualization can increase the effective hardware resource utilization of a PCI-Express device (i.e., the number of applications executing on the device). This approach has been addressed in the Single Root I/O Virtualization (SR-MY) and Sharing Specification, Revision 1.0, Sep. 11, 2007, as well as in the Multiple Root I/O Virtualization (MR-IOV) and Sharing Specification, revision 1.0, May 12, 2008, from the PCI Special Interest Group (SIG), whose disclosures are incorporated herein by reference. Both the SR-IOV and MR-IOV specifications define extensions to the PCIe specification, and enable multiple system images to share PCIe hardware resources. A system image comprises computer software such as operating systems, used to execute applications or trusted services, e.g., a shared or non-shared I/O device driver.
SR-IOV and MR-IOV enable a PCIe device to appear to be multiple separate physical PCIe devices. In addition to functions, which comprise PCIe device configurations, SR-IOV and MR-IOV introduce the idea of physical functions and virtual functions, which can be used to enhance performance of the PCIe device.
Physical functions are full-featured PCIe functions (per the PCI Express® Base Specification, Revision 3.0, Oct. 24, 2010, from PCI-SIG, whose disclosure is incorporated herein by reference) that support the SR-IOV capability and are accessible to a single root PCI Manager (which can be part of a multiple root system), a virtual image, or a system image. In addition to having the capability to convey data “in and out” of a PCIe device, physical functions typically have full configuration resources, thereby enabling them to configure or control the PCIe device via the physical functions.
Virtual functions are “lightweight” PCIe function that execute on a SR-IOV/MR-IOV endpoint, and are directly accessible by a system image. Each instance of a virtual function can link to an underlying physical function, and typically only has the ability to convey data in and out of the PCIe device.
The SR-IOV/MR-IOV capability typically reserves 16 bits for the number of virtual functions (i.e., NumVFs located at address 0×10 in the SR-IOV Capability), meaning that the total number of virtual functions can theoretically reach 65,536 (i.e., 64K)